1
00:00:00,265 --> 00:00:02,265
[ELECTRONIC MUSIC]

2
00:00:10,635 --> 00:00:13,787
Hello again, now the short video
that you are about to see

3
00:00:13,908 --> 00:00:18,290
it is an explanation 
about the NER model and the IRA model.

4
00:00:18,290 --> 00:00:22,580
Carlo Eugeni will give us an explanation
about both models

5
00:00:22,580 --> 00:00:25,914
and their main differences.
I hope you will enjoy it.

6
00:00:27,582 --> 00:00:33,737
There are different models
that are used to assess quality.

7
00:00:34,192 --> 00:00:40,815
One is the NER model
based on Word Error Rate system.

8
00:00:41,163 --> 00:00:42,663
Basically...

9
00:00:43,755 --> 00:00:48,287
it is aimed at comparing 
the source text

10
00:00:48,287 --> 00:00:50,287
and the target text,

11
00:00:50,287 --> 00:00:53,606
and see if the words of the target text

12
00:00:53,606 --> 00:00:57,107
correspond to the ones 
in the source text

13
00:00:57,107 --> 00:01:02,595
this model is very good 
for analysing

14
00:01:02,595 --> 00:01:06,416
and assessing verbatim subtitles.

15
00:01:06,416 --> 00:01:09,000
If subtitles are sensatim

16
00:01:12,867 --> 00:01:16,626
the NER model also provide tools

17
00:01:16,626 --> 00:01:20,021
for assessing if...

18
00:01:20,840 --> 00:01:27,692
If the quality of the reformulation
is good or not.

19
00:01:27,692 --> 00:01:30,587
But it is not really developed.

20
00:01:32,770 --> 00:01:39,304
That is why in Italy
or at least in some cases

21
00:01:39,304 --> 00:01:41,304
we use the IRA model

22
00:01:41,304 --> 00:01:46,230
which is idea-unit rendition assessment.

23
00:01:46,230 --> 00:01:48,230
It is based on idea-units.

24
00:01:48,230 --> 00:01:54,385
The rational of the IRA model is...

25
00:01:58,160 --> 00:01:59,677
Does the subtitle

26
00:01:59,677 --> 00:02:04,846
provide the same message
as the source text, the speech?

27
00:02:04,846 --> 00:02:10,591
If yes, then the all subtitle
is considered...

28
00:02:13,426 --> 00:02:14,668
as good.

29
00:02:15,836 --> 00:02:20,747
So it is not based on word
it is based on idea units, on sentences,

30
00:02:21,217 --> 00:02:22,672
or clauses.

31
00:02:24,294 --> 00:02:27,340
So it follows a different rationale

32
00:02:27,340 --> 00:02:33,000
because it sees 
if there has been a message

33
00:02:33,000 --> 00:02:35,136
which has been delivered or not.

34
00:02:35,136 --> 00:02:38,516
For example, in a long sentence
with many adjectives

35
00:02:38,516 --> 00:02:40,016
if you remove one

36
00:02:40,016 --> 00:02:43,456
in the case of the NER model
you have one mistake

37
00:02:43,456 --> 00:02:46,654
in the case of the IRA model
you do not have mistakes

38
00:02:46,654 --> 00:02:51,095
because the message is delivered.

39
00:02:51,095 --> 00:02:52,746
The other way around

40
00:02:52,746 --> 00:02:57,627
if one word is missing
in the IRA model

41
00:02:57,627 --> 00:03:01,992
and this word is the verb for example,

42
00:03:01,992 --> 00:03:07,237
meaning that you are not able
to deliver the message

43
00:03:07,237 --> 00:03:10,844
in the case of NER model
it is one mistake

44
00:03:10,844 --> 00:03:12,526
in the case of the IRA model

45
00:03:12,526 --> 00:03:16,376
it is a all idea-unit 
which is compromised.

46
00:03:17,089 --> 00:03:21,000
They both work very well 

47
00:03:21,000 --> 00:03:26,503
with what they are designed 
to assess,

48
00:03:26,503 --> 00:03:29,792
meaning verbatim subtitles
in the case of the NER model

49
00:03:29,792 --> 00:03:33,885
and sensatim subtitles
in the case of the IRA model.

50
00:03:35,492 --> 00:03:40,343
But I see some limitations in both

51
00:03:40,343 --> 00:03:43,313
because it depends
on how you use them.

52
00:03:44,541 --> 00:03:46,541
[ELECTRONIC MUSIC]